Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Non-relational data storage management mechanism for massive unstructured data
LIU Chao, HU Chengyu, YAO Hong, LIANG Qingzhong, YAN Xuesong
Journal of Computer Applications    2016, 36 (3): 670-674.   DOI: 10.11772/j.issn.1001-9081.2016.03.670
Abstract679)      PDF (819KB)(514)       Save
Traditional relational data storage systems have been criticized by poor performance and lacking of fault tolerance, therefore it cannot satisfy the efficiency requirement of the massive unstructured data management. A non-relational storage management mechanism with high-performance and high-availability was proposed. First, a user-friendly application interface was designed, and data could be distributed to multiple storage nodes through efficient consistent hashing algorithm. Second, a configurable data replication mechanism was presented to enhance availability of the storage system. Finally, a query fault handling mechanism was proposed to improve the storage system's fault-tolerance and avoid service outages, which were caused by the node failure. The experimental results show that the concurrent access capacity of the proposed storage system increases by 30% and 50% respectively compared to traditional file system and relational database under different user workloads; meanwhile, the availability loss of the storage system under the fault state is less than 14% in a reasonable response time. Therefore, it is applicable for efficient storage management of massive unstructured data.
Reference | Related Articles | Metrics
Graph data processing technology in cloud platform
LIU Chao, TANG Zhengwang, YAO Hong, HU Chengyu, LIANG Qingzhong
Journal of Computer Applications    2015, 35 (1): 43-47.   DOI: 10.11772/j.issn.1001-9081.2015.01.0043
Abstract619)      PDF (794KB)(623)       Save

MapReduce computation model can not satisfy the efficiency requirement of graph data processing in the Hadoop cloud platform. In order to address the issue, a novel computation framework of graph data processing, called MyBSP (My Bulk Synchronous Parallel), was proposed. MyBSP is similar with Pregel developed from Google. Firstly, the running mechanism and shortcomings of MapReduce were analyzed. Secondly, the structure, workflow and principal interfaces of MyBSP framework were described. Finally, the principle of the PageRank algorithm for graph data processing was analyzed. Subsequently, the design and implementation of the PageRank algorithm for graph data processing were presented. The experimental results show that, the iteration processing performance of graph data processing algorithm based on the MyBSP framework is raised by 1.9-3 times compared with the algorithm based on MapReduce. Furthermore, the execution time of the MyBSP algorithm is reduced by 67% compared with MapReduce approach. Thus, MyBSP can efficiently meet the application prospect of graph data processing.

Reference | Related Articles | Metrics